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Abstract — We derive a locally projective noise re- 
duction scheme for nonlinear time series using con- 
cepts from deterministic dynamical systems, or chaos 
theory. We will demonstrate its effectiveness with an 
example with known deterministic dynamics and dis- 
cuss methods for the verification of the results in the 
case of an unknown deterministic system. 

I. INTRODUCTION 

The paradigm of deterministic chaos as an alternative 
explanation for complex temporal behaviour has 
made the development of novel signal processing tech- 
niques necessary. Deterministic chaotic systems are 
characterised by exponentially decaying correlations 
and thus broad band power spectra. Thus, except 
for the case of highly oversampled, time continuous 
signals, filtering by frequency cannot be applied since 
signal and noise have similar spectral properties. On 
the other hand, deterministic dynamics of the form 



F(x„ 



(1) 



or given by an ordinary differential equation of first 
order satisfying a Lipschitz condition for unique solu- 
tions, generates strong signatures when viewed in its 
phase space. Nonlinear noise reduction methods have 
been developed to exploit these structures. Concep- 
tual as well as technical issues arising in such a sit- 
uation have been well discussed in the literature, see 
Kostelich and Schreiber |Q for a review containing the 
relevant references. 

In engineering applications, we usually face a differ- 
ent situation — the signals themselves often contain 
a stochastic component and we cannot make the as- 
sumption that the system is of the form (Q) and deter- 
ministic chaos is present. It turns out that at least for 
a subclass of the nonlinear filtering schemes, the phase 
space projection techniques deterministic chaos is 
not a necessary requirement. The only assumption 
that has to be made is that the signal of interest is ap- 
proximately described by a manifold that has a lower 
dimension than some phase space it is embedded in. 



This is formally true for low dimensional determinis- 
tic signals, but also for certain stochastically driven 
nonlinear phenomena. 

In this paper, a phase space projection scheme for 
noise reduction will be motivated and described. We 
will give an example with known deterministic dynam- 
ics for the purpose of illustration. Applications to real 
data are discussed in Rcf. in this volume. 

II. METHOD 

Let {x„} be the states of a system at times n = 
1,...,A'', represented in some vector space 7^™. A 
[m — Q) -dimensional submanifold T of this space can 
be specified by Fq{y) = 0, q — I, . . . ,Q. Suppose 
such a manifold exists such that the sequence of vec- 
tors {x„}, possibly changed by small displacements 
{e„} lies on that surface: 



Fq (X/j 



= 0, 



yq,n. 



(2) 



The quantity (e^) denotes the (root mean squared) 
average error we make by approximating the points 
{x„} by the manifold J-. For a useful approximation 
we require the functions Fq to be smooth and the se- 
quence {e„} to be small in the rms sense. 

In a measurement, we can only obtain noisy data 
y„ = x„ + J7„, where {rj^} is some random contam- 
ination. The manifold J- is not known a priori and 
has to be estimated from the data. By projecting the 
points {yn} onto the estimated manifold J- we can 
aim to recover x^ = x„ -I- e„. If we can find a suitable 
manifold — and carry out the projections — such that 
(e^) < (ry^), then we have in fact reduced the obser- 
vational error: The projected sequence is closer to the 
true states {x„} than the noisy observations. 

In all the cases discussed in this paper, only a 
scalar measurement of the system states is available: 
Sn = s(x„), n = 1,...,7V. A multi-dimensional 
vector representation can be obtained by consider- 
ing also time delayed copies of the scalar sequence: 
s„ = (s„_(„_i)^, Sn-{m-2)T, Sn)- For determinis- 
tic dynamical systems, theorems are available ^ on 



neighbourhood, we compute the local mean 




Figure 1: Illustration of the local projection scheme. 
For each point to be corrected, a neighbourhood is 
formed (grey shaded area), the point cloud in which 
is then approximated by an ellipsoid. An approxi- 
mately two-dimensional manifold embedded in a three- 
dimensional space could for example be cleaned by 
projecting onto the first two principal directions. 



the equivalence of the sequences {s„} and {x„}. Sup- 
pose a system is governed by deterministic equations 
of motion in to— 1-dimensional delay coordinate space. 
Then Eq.(j|) becomes 



Sn - F{Sn-(d-l)T, ■ ■■, S(n-r)) = 0, 



Vn . 



(3) 



This means that in m-dimensional embedding space, 
there exists an m — 1-dimensional manifold contain- 
ing the signal. Noisy measurements of such processes 
can be cleaned by imposing the relation (||) on the 
data. In higher dimensional embeddings, more than 
one independent equations of the form (||) exist. Still, 
these functions Fq are unknown and have to be ap- 
proximated by a fit. 

In time series work, the most practical way to ap- 
proximate data by a manifold is by a locally linear 
representation. It should in principle be possible to 
fit global nonlinear constraints Fq from data, but the 
problem is complicated by the necessity to have Q lo- 
cally independent equations. In the locally linear case 
this is achieved by establishing local principal compo- 
nents. The derivation will not be repeated here, it is 
carried out for example in Ref. Q]. The resulting al- 
gorithm proceeds as follows. In an embedding space 
of dimension m we form delay vectors s„. For each 
of these we construct small neighborhoods Un , so that 
the neighbouring points are Sfc,fc e Un- Within each 
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and the (m x to) covariance matrix 



(4) 



(5) 



It has been found advantageous [|| to introduce a diag- 
onal weight matrix R and define a transformed version 



of the covariance matrix Ti 



RiiCijRjj for the calcu- 



lation of the principal directions. In order to penalise 
corrections based on the first and last coordinates in 
the delay window one puts Rqq = Rmm = r where r is 
large. The other values on the diagonal of R are 1. 

The eigenvectors of the matrix are the semi- 
axes of the best approximating ellipsoid of the cloud 
of points. These are local versions of the well known 
principal components, or singular vectors, see for ex- 
ample Refs. 1^ . If the clean data lives near a smooth 
manifold with dimension toq < to, and if the variance 
of the noise is sufficiently small for the linearisation to 
be valid, then for the noisy data the covariance matrix 
will have large eigenvalues spanning the smooth man- 
ifold and small eigenvalues in all other directions. Of 
course, this is strictly true only if the neighbourhoods 
are larger than the noise level. In practice, a tradeoff 
between the clear definition of the noise directions and 
a good linear approximation has to be balanced. 

By projecting onto the subspace of large eigenvec- 
tors, we move the vector under consideration towards 
the manifold. The procedure is graphically illustrated 
in Fig. ^. The correction is done for each embedding 
vector, resulting in a set of corrected vectors in em- 
bedding space. Since each element of the scalar time 
series occurs in m different embedding vectors, we fi- 
nally have as many different suggested corrections, of 





Figure 2: Local linear approximation to a one- 
dimensional curve. Left: approximations are not tan- 
gents but secants and all the centres of mass (crosses) 
of different neighbourhoods arc shifted inward with re- 
spect to the curvature. Right: a tangent approxima- 
tion is obtained by shifting the centre of mass outward 
with respect to the curvature. The open square de- 
notes the average of the centres of mass of adjacent 
neighbourhoods, the filled square is the corrected cen- 
tre of mass. 





Figure 3: Ikeda map example. Upper left: noise 
free data; right: data contaminated with 5% additive 
Gaussian white noise. Lower left: data after low-pass 
filtering; right: data after nonlinear noise reduction. 



which we simply take the average. Therefore in em- 
bedding space the corrected vectors do not precisely 
lie on the local subspaces but are only moved towards 
them. 

If the local linear subspaces are determined in the 
way outlined above, they are not really tangent to the 
curved manifold but rather intersect with it, as illus- 
trated in Fig. ^. Therefore it is preferable to use a 
corrected centre of mass s*-"-* given by 



=(«) _ („) 



1 



E 

fcew„ 



(6) 



This correction prevents a bias towards corrections in 
the main direction of curvature. If it is omitted, and 
if rather large neighbourhoods are used, the set of em- 
bedded data points in phase space may be subject to 
an overall contraction on multiple iterations of the pro- 
cedure. 

A computer program that implements the scheme 
described in this paper is included in the TISEAN soft- 
ware project and is publicly available j^. 

III. NUMERICAL EXAMPLE 

Let us show an example for the noise reduction ca- 
pability of the algorithm for deterministically chaotic 
time series. The Ikeda map is given by the formula 




Zn+i = 1 + 0.9z„ exp 0.4i 



(7) 



Figure 4: Local slopes of the correlation sum indi- 
cating fractal structure (Ikeda map example). Upper: 
noise free data. Middle: data contaminated with 5% 
additive Gaussian white noise. Lower: after nonlinear 
noise reduction. See text. 



where {zn} is a sequence of complex numbers. Con- 
sider as a model time series the sequence Xn = '^{zn) 
given by the real parts of Zn- In Fig. ^, the up- 
per left panel shows a delay representation with de- 
lay T = 1 of the noise free sequence {xn\. In order 
to mimic a simplified experimental situation, the se- 
quence Sn — Xn-\- Tin has bccu Contaminated by adding 
Gaussian independent random numbers of a rms am- 
plitude of 5% of that of the clean data, upper right 
panel. The lower row shows two different attempts on 
noise reduction. Left, a low pass filter has been ap- 
plied to suppress the highest 4% of frequencies in the 
Nyquist interval. Since the signal has still significant 
power at these frequencies, the filter is inadequate and, 
in fact, severely distorts the phase space structure. In 
the right panel, nonlinear noise reduction has been ap- 
plied using embeddings in to = 7 dimensions and local 
projections onto to — Q = 3 dimensions. The neigh- 
bourhood size was taken to be 0.02 units, which is 
about the absolute noise level added. The figure is the 
result of three iterations. The error was reduced by a 
factor of 1.7 in terms of rms amplitude. Note however 
that the data is probably much closer than that to a 



true trajectory of the Ikeda system. 

In Fig. ^, the effect of noise reduction on this frac- 
tal attractor is demonstrated with the help of the 
Grassberger-Procaccia correlation sum, C(e), the frac- 
tion of pairs of points closer than e in delay coordinate 
space. As explained for example in Ref. Q, we take 
the local slope in a double logarithmic plot of C(e) 
versus e as an effective scale dependent scaling expo- 
nent d{e) = dlogC(e)/dloge. If a significant plateau 
occurs and certain precautions have been taken, the 
plateau value of (i(e) would estimate the correlation di- 
mension of the attractor underlying the data. Indeed, 
such a plateau can be seen for the noise free data (up- 
per panel) while it is rather small for the noisy data. 
After nonlinear noise reduction (lower panel), the scal- 
ing is recovered down to much smaller length scales. 

IV. DISCUSSION 

With the emergence of experiments exhibiting deter- 
ministic chaos, nonlinear filtering techniques became 
a necessity. The earliest approaches required detailed 
understanding of the dynamics and the stability struc- 
ture in phase space. The phase space projection tech- 
nique described in this paper goes back to Ref. but 
some earlier techniques are quite similarly set-up 
These techniques do not require a detailed model, or 
a global fit of such a model, for the dynamics. Rather, 
the inhomogeneous distribution in reconstructed phase 
space is enhanced by projecting onto local linear sub- 
spaces. We have demonstrated in this paper that the 
method is effective in reducing noise in an artificial, 
chaotic example. Applications to real time series data 
will be discussed in Ref. Q in this volume. 
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